Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors.

OBJECTIVES: To investigate the impact of potential risk of bias elements on effect estimates in randomized trials. STUDY DESIGN AND SETTING: We conducted a systematic survey of meta-epidemiological studies examining the influence of potential risk of bias elements on effect estimates in randomized trials. We included only meta-epidemiological studies that either preserved the clustering of trials within meta-analyses (compared effect estimates between trials with and without the potential risk of bias element within each meta-analysis, then combined across meta-analyses; between-trial comparisons), or preserved the clustering of substudies within trials (compared effect estimates between substudies with and without the element, then combined across trials; within-trial comparisons). Separately for studies based on between- and within-trial comparisons, we extracted ratios of odds ratios (RORs) from each study and combined them using a random-effects model. We made overall inferences and assessed certainty of evidence based on Grading of Recommendations, Assessment, development, and Evaluation and Instrument to assess the Credibility of Effect Modification Analyses. RESULTS: Forty-one meta-epidemiological studies (34 of between-, 7 of within-trial comparisons) proved eligible. Inadequate random sequence generation (ROR 0.94, 95% confidence interval [CI] 0.90-0.97) and allocation concealment (ROR 0.92, 95% CI 0.88-0.97) probably lead to effect overestimation (moderate certainty). Lack of patients blinding probably overestimates effects for patient-reported outcomes (ROR 0.36, 95% CI 0.28-0.48; moderate certainty). Lack of blinding of outcome assessors results in effect overestimation for subjective outcomes (ROR 0.69, 95% CI 0.51-0.93; high certainty). The impact of patients or outcome assessors blinding on other outcomes, and the impact of blinding of health-care providers, data collectors, or data analysts, remain uncertain. Trials stopped early for benefit probably overestimate effects (moderate certainty). Trials with imbalanced cointerventions may overestimate effects, while trials with missing outcome data may underestimate effects (low certainty). Influence of baseline imbalance, compliance, selective reporting, and intention-to-treat analysis remain uncertain. CONCLUSION: Failure to ensure random sequence generation or adequate allocation concealment probably results in modest overestimates of effects. Lack of patients blinding probably leads to substantial overestimates of effects for patient-reported outcomes. Lack of blinding of outcome assessors results in substantial effect overestimation for subjective outcomes. For other elements, though evidence for consistent systematic overestimate of effect remains limited, failure to implement these safeguards may still introduce important bias.

Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues.

OBJECTIVES: To establish whether items included in instruments published in the last decade assessing risk of bias of randomized controlled trials (RCTs) are indeed addressing risk of bias. STUDY DESIGN AND SETTING: We searched Medline, Embase, Web of Science, and Scopus from 2010 to October 2021 for instruments assessing risk of bias of RCTs. By extracting items and summarizing their essential content, we generated an item list. Items that two reviewers agreed clearly did not address risk of bias were excluded. We included the remaining items in a survey in which 13 experts judged the issue each item is addressing: risk of bias, applicability, random error, reporting quality, or none of the above. RESULTS: Seventeen eligible instruments included 127 unique items. After excluding 61 items deemed as clearly not addressing risk of bias, the item classification survey included 66 items, of which the majority of respondents deemed 20 items (30.3%) as addressing risk of bias; the majority deemed 11 (16.7%) as not addressing risk of bias; and there proved substantial disagreement for 35 (53.0%) items. CONCLUSION: Existing risk of bias instruments frequently include items that do not address risk of bias. For many items, experts disagree on whether or not they are addressing risk of bias.

Development of a Novel Communication Liaison Program to Support COVID-19 Patients and Their Families.

In the spring of 2020, our hospital faced a surge of critically ill coronavirus disease 2019 patients, with intensive care unit (ICU) occupancy peaking at 204% of the baseline maximum capacity. In anticipation of this surge, we developed a remote communication liaison program to help the ICU and palliative care teams support families of critically ill patients. In just nine days from inception until implementation, we recruited and prepared ambulatory specialty providers to serve in this role effectively, despite minimal prior critical care experience. We report here the primary elements needed to reproduce and scale this program in other hospitals facing similar ICU surges, including a checklist for replication (Appendix I). Keys to success include strong logistical support, clinical reference material designed for rapid evolution, and a liaison team structure with peer coaching.

Impact of clinical training on recruiting graduating health professionals.

OBJECTIVES: Recruiting professional staff is an important business reason for hospitals allowing health trainees to engage in supervised patient care. Whereas prior studies have focused on educational institutions, this study focuses on teaching hospitals and whether trainees' clinical experiences affect their willingness to work (ie, recruitability) for the type of healthcare center where they trained. STUDY DESIGN: A pre-post, observational study based on Learners' Perceptions Survey data in which respondents served as their own controls. METHODS: Convenience sample of 15,207 physician, 11,844 nursing, and 13,012 associated health trainees who rotated through 1 of 169 US Department of Veterans Affairs (VA) medical centers between July 1, 2014, and June 30, 2017. Generalized estimating equations computed how clinical, learning, working, and cultural experiences influenced pre-post differences in willingness to consider VA for future employment. RESULTS: VA recruitability increased dramatically from 55% pretraining to 75% post training (adjusted odds ratio [OR], 2.1; 95% CI, 2.0-2.1; P <.001) in all 3 cohorts: physician (from 39% to 59%; OR, 1.6; 95% CI, 1.5-1.6; P <.001), nursing (from 61% to 84%; OR, 2.5; 95% CI, 2.4-2.6; P <.001), and associated health trainees (from 68% to 87%; OR, 2.7; 95% CI, 2.6-2.9; P <.001). For all trainees, changes in recruitability (P <.001) were associated with how trainees rated their clinical learning environment, personal experiences, and culture of psychological safety. Satisfaction ratings with faculty and preceptors (P <.001) were associated with positive changes in recruitability among nursing and associated health students but not physician residents, whereas nursing students who gave higher ratings for interprofessional team culture became less recruitable. CONCLUSIONS: Academic medical centers can attract their health trainees for future employment if they provide positive clinical, working, learning, and cultural experiences.

Has Interprofessional Education Changed Learning Preferences? A National Perspective.

OBJECTIVE: To assess how changes in curriculum, accreditation standards, and certification and licensure competencies impacted how medical students and physician residents value interprofessional team and patient-centered care. PRIMARY DATA SOURCE: The Department of Veterans Affairs Learners' Perceptions Survey (2003-2013). The nationally administered survey asked a representative sample of 56,569 U.S. medical students and physician residents, with a comparison group of 78,038 nonphysician trainees, to rate satisfaction with 28 elements, in two overall domains, describing their clinical learning experiences at VA medical centers. STUDY DESIGN: Value preferences were scored as independent adjusted associations between an element (interprofessional team, patient-centered preceptor) and the respective overall domain (clinical learning environment, faculty, and preceptors) relative to a referent element (quality of clinical care, quality of preceptor). PRINCIPAL FINDINGS: Physician trainees valued interprofessional (14 percent vs. 37 percent, p < .001) and patient-centered learning (21 percent vs. 36 percent, p < .001) less than their nonphysician counterparts. Physician preferences for interprofessional learning showed modest increases over time (2.5 percent/year, p < .001), driven mostly by internal medicine and surgery residents. Preferences did not increase with trainees' academic progress. CONCLUSIONS: Despite changes in medical education, physician trainees continue to lag behind their nonphysician counterparts in valuing experience with interprofessional team and patient-centered care.

Does Psychological Safety Impact the Clinical Learning Environment for Resident Physicians? Results From the VA's Learners' Perceptions Survey.

BACKGROUND: Psychological safety (PS) is the perception that it is safe to take interpersonal risks in the work environment. In teaching hospitals, PS may influence the clinical learning environment for trainees. OBJECTIVE: We assessed whether resident physicians believe they are psychologically safe, and if PS is associated with how they rate satisfaction with their clinical learning experience. METHODS: Data were extracted from the Learners' Perceptions Survey (LPS) of residents who rotated through a Department of Veterans Affairs health care facility for academic years 2011-2014. Predictors of PS and its association with resident satisfaction were adjusted to account for confounding and response rate biases using generalized linear models. RESULTS: The 13 044 respondents who completed the LPS (30% response rate) were comparable to nonpediatric, non-obstetrics-gynecology residents enrolled in US residency programs. Among respondents, 11 599 (89%) agreed that ". . . members of the clinical team of which I was part are able to bring up problems and tough issues." Residents were more likely to report PS if they were male, were in a less complex clinical facility, in an other medicine or psychiatry specialty, or cared for patients who were aged, had multiple illnesses, or had social supports. Nonpsychiatric residents felt safer when treating patients with no concurrent mental health diagnoses. PS was strongly associated with how residents rated their satisfaction across 4 domains of their clinical learning experience (P < .001). CONCLUSIONS: PS appears to be an important factor in resident satisfaction across 4 domains that evaluators of graduate medical education programs should consider when assessing clinical learning experiences.

Redesigning Diabetes Care: Defining the Role of Endocrinologists Among Alternative Providers.

The current diabetes epidemic threatens to overwhelm the healthcare system unless we redesign how diabetes care is delivered. The number of endocrinologists is grossly inadequate to provide care for all individuals with diabetes, but with the appropriate utilization of the primary care workforce and alternative healthcare providers working together in teams, effective diabetes care can be provided to all. We propose a patient-centered, goal-based approach with resources devoted to care coordination, measurement of outcomes, appropriate use of technology, and measurement of patient satisfaction. Financial incentives to healthcare systems and providers need to be based on defined outcome measures and reducing long-term total medical expenditures, rather than reimbursement based on number of visits and lengthy documentation. Endocrinologists have a responsibility in setting up effective diabetes care delivery systems within their organizations, in addition to delivering diabetes care and serving as a resource for the educational needs for other medical professionals in the community. There are major challenges to implementing such systems, both at the financial and organizational levels. We suggest a stepwise implementation of discrete components based on the local priorities and resources and provide some examples of steps we have taken at our institution.

The learners' perceptions survey-primary care: assessing resident perceptions of internal medicine continuity clinics and patient-centered care.

BACKGROUND: In 2010, the Department of Veterans Affairs (VA) implemented a national patient-centered care initiative that organized primary care into interdisciplinary teams of health care professionals to provide patient-centered, continuous, and coordinated care. OBJECTIVE: We assessed the discriminate validity of the Learners' Perceptions Survey-Primary Care (LPS-PC), a tool designed to measure residents' perceptions about their primary and patient-centered care experiences. METHODS: Between October 2010 and June 2011, the LPS-PC was administered to Loma Linda University Medical Center internal medicine residents assigned to continuity clinics at the VA Loma Linda Healthcare System (VALLHCS), a university setting, or the county hospital. Adjusted differences in satisfaction ratings across settings and over domains (patient- and family-centered care, faculty and preceptors, learning, clinical, work and physical environments, and personal experience) were computed using a generalized linear model. RESULTS: Our response rate was 86% (77 of 90). Residents were more satisfied with patient- and family-centered care at the VALLHCS than at either the university or county (P < .001). However, faculty and preceptors (odds ratio [OR]  =  1.53), physical (OR  =  1.29), and learning (OR  =  1.28) environments had more impact on overall resident satisfaction than patient- and family-centered care (OR  =  1.08). CONCLUSIONS: The LPS-PC demonstrated discriminate validity to assess residents' perceptions of their patient-centered clinical training experience across outpatient primary care settings at an internal medicine residency program. The largest difference in scores was the patient- and family-centered care domain, in which residents rated the VALLHCS much higher than the university or county sites.

Training satisfaction for subspecialty fellows in internal medicine: findings from the Veterans Affairs (VA) Learners' Perceptions Survey.

BACKGROUND: Learner satisfaction assessment is critical in the design and improvement of training programs. However, little is known about what influences satisfaction and whether trainee specialty is correlated. A national comparison of satisfaction among internal medicine subspecialty fellows in the Department of Veterans Affairs (VA) provides a unique opportunity to examine educational factors associated with learner satisfaction. We compared satisfaction across internal medicine fellows by subspecialty and compared factors associated with satisfaction between procedural versus non-procedural subspecialty fellows, using data from the Learners' Perceptions Survey (LPS), a validated survey tool. METHODS: We surveyed 2,221 internal medicine subspecialty fellows rotating through VA between 2001 and 2008. Learners rated their overall training satisfaction on a 100-point scale, and on a five-point Likert scale ranked satisfaction with items within six educational domains: learning, clinical, working and physical environments; personal experience; and clinical faculty/preceptor. RESULTS: Procedural and non-procedural fellows reported similar overall satisfaction scores (81.2 and 81.6). Non-procedural fellows reported higher satisfaction with 79 of 81 items within the 6 domains and with the domain of physical environment (4.06 vs. 3.85, p <0.001). Satisfaction with clinical faculty/preceptor and personal experience had the strongest impact on overall satisfaction for both. Procedural fellows reported lower satisfaction with physical environment. CONCLUSIONS: Internal medicine fellows are highly satisfied with their VA training. Nonprocedural fellows reported higher satisfaction with most items. For both procedural and non-procedural fellows, clinical faculty/preceptor and personal experience have the strongest impact on overall satisfaction.

Studying the effects of ACGME duty hours limits on resident satisfaction: results from VA learners' perceptions survey.

BACKGROUND: As the Accreditation Council on Graduate Medical Education (ACGME) deliberates over further limiting duty hours of graduate medical education (GME) trainees, few large-scale studies have shown residents to be satisfied with the effect the 2003 standards have had on clinical care, education outcomes, or working environments. This study measures the effect of the 2003 duty hours limits on resident-reported satisfaction with GME training during their rotations through the Department of Veterans Affairs (VA) medical centers from 2001 through 2007. METHOD: Self-reported satisfaction with clinical care and education environments were assessed by comparing responses to VA's annual Learners' Perceptions Survey administered before 2003 with responses administered after 2003. To measure duty hours effects on satisfaction, before-after differences were adjusted for covariate biases modeled after an exhaustive covariate search with 10-fold cross-validation. Because nonteaching controls are not available in satisfaction studies, we used a robust differencing variable technique to control before-after differences for trend biases in the simultaneous presence of missing data and possible model misspecification. RESULTS: There were 19,605 responders. Adjusting for covariate and trend biases, after the 2003 ACGME standards, 25% more residents in medicine specialties reported satisfaction with VA clinical environment and 11% more with VA preceptors and faculty. For surgery, 33% more residents reported satisfaction with VA clinical environment and 12% more with VA preceptors and faculty. Satisfaction with working environment was mixed. CONCLUSIONS: The 2003 ACGME duty hours standards were associated with improved satisfaction for resident clinical training and learning environments.

Measuring the intensity of resident supervision in the department of veterans affairs: the resident supervision index.

PURPOSE: To develop a survey instrument designed to quantify supervision by attending physicians in nonprocedural care and to assess the instrument's feasibility and reliability. METHOD: In 2008, the Department of Veterans Affairs (VA) Office of Academic Affiliations convened an expert panel to adopt a working definition of attending supervision in nonprocedural patient care and to construct a survey to quantify it. Feasibility was field-tested on residents and their supervising attending physicians at primary care internal medicine clinics at the VA Loma Linda Healthcare System in their encounters with randomly selected outpatients diagnosed with either major depressive disorder or diabetes. The authors assessed both interrater concurrent reliability and test-retest reliability. RESULTS: The expert panel adopted the VA's definition of resident supervision and developed the Resident Supervision Index (RSI) to measure supervision in terms of residents' case understanding, attending physicians' contributions to patient care through feedback to the resident, and attending physicians' time (minutes). The RSI was field-tested on 60 residents and 37 attending physicians for 148 supervision episodes from 143 patient encounters. Consent rates were 94% for residents and 97% for attending physicians; test-retest reliability intraclass correlations (ICCs) were 0.93 and 0.88, respectively. Concurrent reliability between residents' and attending physicians' reported time was an ICC of 0.69. CONCLUSIONS: The RSI is a feasible and reliable measure of resident supervision that is intended for research studies in graduate medical education focusing on education outcomes, as well as studies assessing quality of care, patient health outcomes, care costs, and clinical workload.

Measuring progressive independence with the resident supervision index: theoretical approach.

BACKGROUND: Graduate medical education is based on an on-the-job training model in which residents provide clinical care under supervision. The traditional method is to offer residents graduated levels of responsibility that will prepare them for independent practice. However, if progressive independence from supervision exceeds residents' progressive professional development, patient outcomes may be at risk. Leaders in graduate medical education have called for "optimal" supervision, yet few studies have conceptually defined what optimal supervision means and whether optimal care is theoretically compatible with progressive independence, nor have they developed a test for progressive independence. OBJECTIVE: This research develops theory and analytic models as part of the Resident Supervision Index to quantify the intensity of supervision. METHODS: We introduce an explicit set of assumptions for an ideal patient-centered theory of optimal supervision of resident-provided care. A critical assumption is that informed attending staff will use available resources to optimize patient outcomes first and foremost, with residents gaining clinical competencies by contributing to optimal care. Next, we derive mathematically the consequences of these assumptions as theoretical results. RESULTS: Under optimal supervision, (1) patient outcome is expected to be no worse than if residents were not involved, (2) supervisors will avoid undersupervising residents (when patients are at increased risk for poor outcomes) or oversupervising residents (when residents miss clinical opportunities to practice care), (3) optimal patient outcomes will be compatible with progressive independence, (4) progressive development can be inferred from progressive independence whenever residents contribute to patient care, and (5) analytic models that test for progressive independence will emphasize adjusting the association between length of graduate medical education training and supervision for case complexity and clinic workload, but not patient health outcomes. CONCLUSION: An explicit theoretical framework is critical to measure scientifically progressive independence from supervision using graduate medical education data.

Measuring progressive independence with the resident supervision index: empirical approach.

BACKGROUND: A Resident Supervision Index (RSI) developed by our research team quantifies the intensity of resident supervision in graduate medical education, with the goal of testing for progressive independence. The 4-part RSI method includes a survey instrument for staff and residents (RSI Inventory), a strategy to score survey responses, a theoretical framework (patient centered optimal supervision), and a statistical model that accounts for the presence or absence of supervision and the intensity of patient care. METHODS: The RSI Inventory data came from 140 outpatient encounters involving 57 residents and 37 attending physicians during a 3-month period at a Department of Veterans Affairs outpatient clinic. Responses are scored to quantitatively measure the intensity of resident supervision across 10 levels of patient services (staff is absent, is present, participated, or provided care with or without a resident), case discussion (resident-staff interaction), and oversight (staff reviewed case, reviewed medical chart, consulted with staff, or assessed patient). Scores are analyzed by level and for patient care using a 2-part model (supervision initiated [yes or no] versus intensity once supervision was initiated). RESULTS: All resident encounters had patient care supervision, resident oversight, or both. Consistent with the progressive independence hypothesis, residents were 1.72 (P  =  .019) times more likely to be fully responsible for patient care with each additional postgraduate year. Decreasing case complexity, increasing clinic workload, and advanced nonmedical degrees among attending staff were negatively associated with supervision intensity, although associations varied by supervision level. CONCLUSIONS: These data are consistent with the progressive independence hypothesis in graduate medical education and offer empirical support for the 4-part RSI method to quantify the intensity of resident supervision for research, program evaluation, and resident assessment purposes. Before informing policy, however, more scientific research in actual teaching settings is needed to better understand the relationships among patient outcomes, clinic workload, case complexity, and graduate medical education experience in resident supervision and professional development.

How to teach evidence-based medicine to teachers: reflections from a workshop experience.

RATIONALE, AIMS AND OBJECTIVES: To summarize 20-year experience of conducting a workshop designed for educators who wish to improve their teaching skills of evidence based medicine (EBM). The goal is to provide tips for educators interested in replicating this educational model. METHODS: Qualitative description of factors associated with the success of the workshop. RESULTS: The factors considered by instructors to be most helpful are: the small group interactive design, role-play and simulation of real world learning environments, a mentorship model and high educator to learner ratio. CONCLUSIONS: Although this experience is observational and does not represent high quality evidence, certain attributes in the design of EBM workshops may lead to better dissemination of EBM concepts. Educators may consider empirically applying some of these attributes and testing their efficacy in comparative studies.

Factors determining medical students' and residents' satisfaction during VA-based training: findings from the VA Learners' Perceptions Survey.

PURPOSE: To compare medical students' and physician residents' satisfaction with Veterans Affairs (VA) training to determine the factors that were most strongly associated with trainee satisfaction ratings. METHOD: Each year from 2001 to 2006, all medical students and residents in VA teaching facilities were invited to complete the Learners' Perceptions Survey. Participants rated their overall training satisfaction on a 100-point scale and ranked specific satisfaction in four separate educational domains (learning environment, clinical faculty, working environment, and physical environment) on a five-point Likert scale. Each domain was composed of unique items. RESULTS: A total of 6,527 medical students and 16,583 physician residents responded to the survey. The overall training satisfaction scores for medical students and physician residents were 84 and 79, respectively (P < .001), with significant differences in satisfaction reported across the training continuum. For both medical students and residents, the rating of each of the four educational domains was statistically significantly associated with the overall training satisfaction score (P < .001). The learning environment domain had the strongest association with overall training satisfaction score, followed by the clinical preceptor, working environment, and physical environment domains; no significant differences were found between medical students and physician residents in the rank order. Satisfaction with quality of care and faculty teaching contributed significantly to training satisfaction. CONCLUSIONS: Factors that influence training satisfaction were similar for residents and medical students. The domain with the highest association was the learning environment; quality of care was a key item within this domain.

Evidence based clinical practice: a primer for urologists.

PURPOSE: Evidence based clinical practice has been defined as the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients. It refers to a broad set of principles and methods intended to ensure that medical decisions, guidelines and health policy are based on well designed studies of therapeutic effectiveness and benefit. MATERIALS AND METHODS: We review the principles and practice of evidence based clinical practice using examples from the urology literature. We further provide a guide to currently available web based evidence based clinical practice resources and guidelines for urologists. RESULTS: Evidence based clinical practice integrates a hierarchy of evidence and patient values with practitioner judgment to guide decision making for the individual patient. Important steps in the evidence based clinical practice process include the formulation of an answerable question and a systematic search of the literature. In the absence of pre-appraised evidence or disease specific guidelines, the practice of evidence based clinical practice relies heavily on the evaluation of the primary literature by the individual urologist. Depending on the question domain (therapy/prevention, etiology/cause/harm, diagnosis or prognosis) and study design, a given study is critically appraised for validity, impact and applicability. Evidence is then integrated with clinical judgment, and patient circumstances and preferences. Finally, the practice of evidence based clinical practice includes a self-assessment of provider performance. CONCLUSIONS: Knowledge, practice and documentation of evidence based clinical practice are of increasing importance to every urologist. Urologists should embrace evidence based clinical practice principles by acquiring the necessary skills to critically appraise the literature for the best evidence applicable to patient care.

Behind closed doors: management of patient expectations in primary care practices.

BACKGROUND: Managed care restrictions on resource use may affect communication between patients and health care professionals. METHODS: To characterize negotiations between primary care physicians and patients with expectations for new medications, tests, or referrals, this observational study combined survey data with audiotape recordings of clinical encounters. Fifty-five physicians from 20 randomly selected primary care practices in a managed care network and 211 patients who voiced specific expectations in a previsit survey were included. From the recorded clinic visits we determined modes of negotiation of patient expectations and requests. From the surveys we determined patient previsit expectations, postvisit fulfillment of expectations, satisfaction, and trust. RESULTS: Two-hundred fifty-six self-reported expectations were captured in 200 audiotape-recorded encounters. Of the previsit expectations, 97.3% were discussed during the encounter. Expectations were expressed by direct patient request (40.6%), mentioning of symptoms related to request (29.7%), or physician-initiated discussion (27.0%). Most expectations were met (66.8%); physicians suggested an alternative 21.6% of the time. Expectations for medications and tests were met more frequently than expectations for referrals (75.6% and 71.4% vs 40.8%). Patient satisfaction and trust remained high regardless of whether expectations were met. Physicians reported that they would not have ordered 62 (44.9%) of 138 requests had the patients not directly asked, and they were uncomfortable filling 8 requests (12.9%). CONCLUSIONS: Previsit expectations for medications, tests, or referrals were discussed at the visit, and physicians met or offered alternatives for nearly 90%. Patients generally received what they asked for and altered physician behavior nearly half of the time.